Virtual Reality (VR) is an emerging “killer application” that has the potential to radically transform existing ways of doing various tasks. A 360-degree panoramic video is captured and used to create a computer-modeled 3D space. Then a user wearing special goggles such as a Head-Mounted-Display (HMD) can actively select and vary his viewpoint to get an immersive experience.
A wide variety of interesting and useful applications are possible as VR camera technology improves and shrinks. A helmet cam such as a GoPro camera could be replaced by a VR panorama camera set to allow the capture of 360-degree panoramas while engaging in various sports activities such as mountain biking, skiing, skydiving, traveling, etc. A VR camera could also be placed on an aerial drone, to allow for VR modeling of an aerial inspection of a construction site, or for travel blogging or video surveillance. A VR camera placed at a family gathering could allow a remote relative to be immersed in the family event using a VR headset. A VR camera on a self-driving car or on a drone could provide input to auto-driving or auto-flying control systems.
How the 360-degree panoramic video is captured and generated can affect the quality of the VR experience. When multiple cameras are used, regions where two adjacent camera images intersect often have visual artifacts and distortion such as ghosting that can mar the user experience, or even give the user a headache !
FIG. 1 shows a prior-art VR ring camera. Ring camera 10 has multiple cameras 12 arranged in a ring. This arrangement of cameras 12 allows for a 360-degree panorama to be captured. When cameras 12 are video cameras, a panoramic video is captured. The Google Jump is an example of a VR ring camera.
FIGS. 2A-C highlight ghosting artifacts created by parallax errors where images from two adjacent cameras are stitched together. In FIG. 2A, cameras 12L, 12R are two adjacent cameras in ring camera 10 of FIG. 1. Object 14 is captured by both cameras 12L, 12R. However, since object 14 is a different distance and angle to each of cameras 12L, 12R, each camera 12L, 12R sees object 14 at a different location on image frame 16.
In FIG. 2B, object 14 may appear on image fame 16 as two different objects 14L, 14R seen by cameras 12L, 12R. Image processing software may attempt to estimate the depth of object 14 relative to each of cameras 12L, 12R to correct the parallax error, but depth estimation is inexact and challenging. This object matching and depth estimation may result in non-linear warping of images. As FIG. 2C shows, distortion may be especially visible near interfaces where adjacent images 18L, 18R are stitched together. The test pattern is distorted at the interface between images 18L, 18R. Square boxes are squished and narrowed at the interface. This ghosting is undesirable.
What is desired is a Virtual Reality (VR) panorama generator that reduces or eliminates ghosting artifacts at interfaces where images from adjacent cameras are stitched together. A panorama generator that does not require depth estimation is desirable. A panorama generator that places high-resolution images over a low-resolution panoramic image is desired to eliminate stitching regions and ghosting artifacts.